There Is No Preview Available For This Item

This item does not appear to have any files that can be experienced on Archive.org.
Please download files in this item to interact with them on your computer.
Show all files

Twitter Iran Dataset

by: Twitter

Publication date: 2018-10-17
Item Size: 65.9G

This dataset was released by Twitter on October 17, 2018 in order to provide transparency for state sponsored propaganda that was alleged to have occurred on their platform in the lead up to and directly following the 2016 Presidential Election. They released two separate datasets one focused on content believed to originate in Iran and another from the Russian Internet Research Agency. More about the release from Twitter can be read here: https://blog.twitter.com/official/en_us/topics/company/2018/enabling-further-research-of-information-operations-on-twitter.html

This Internet Archive item represents the data for the Iran dataset that was downloaded on October 17 from links on the Twitter website using the wget command line utility. You can see the specific URLs that were downloaded in the ira/urls.txt file.

The Twitter website linked to https://storage.cloud.google.com/twitter-election-integrity/hashed/ira/ira_media_file_list_hashed.txt which required a Google login. So this URL was updated to be https://storage.googleapis.com/twitter-election-integrity/hashed/ira/ira_media_file_list_hashed.txt to match the others, which did not require a login.

The only post-processing that was down after downloading the referenced data was to download the zip file URLs contained in ira_media_file_list_hashed.txt which are compressed archives of media content not directly available in the initially downloaded data.

A static snapshot of the webpages announcing the availability of the data can be found in the twitter-data-staticsite.zip file that was included above the ira directory. You should be able to unzip this file and place it in a directory that a webserver will deliver content from, to view the site. Or you can view it in the Wayback Machine.

What follows here is the documentation Twitter provided in Twitter_Elections_Integrity_Datasets_hashed_README.txt:

---

Twitter Elections Integrity Datasets Readme

Version 1.0
October 16, 2018

For information about Twitter’s election integrity efforts, as well as updates to these d
atasets, please see https://about.twitter.com/en_us/values/elections-integrity.html#data.

I. Dataset Structure Description

Each dataset includes 2 folders, which indicate the organizations the user accounts are from:

ira
iranian

Each folder includes 6 types of files / folder, comprising the full archive of informatio
n disclosed about that information operation:

tweets_csv_hashed.zip - full tweets and metadata
users_csv_hashed.zip - list of users and profile metadata
profile_banner_hashed.zip - profile photos and profile banners. Users with the default Twitter profile pic and/or banner are not included
tweet_media_hashed - folder containing tweet media. Within the folder, there are a number of .zip files where each contains a number of users' tweet media (Number of .zip files varies based on the volume of media; See tweet_media_hashed_README files for details about which users are in which numbered .zip file)
tweet_media_hashed_README - file that details which users' tweet media are in which numbered .zip file
periscope_hashed.zip - Periscope broadcasts, where each sub-folder contains the users' broadcasts (users without a Periscope account are not included; users with a Periscope account with no broadcasts have an empty sub-folder)

II. Tweet Dataset Field Descriptions

csv files in _tweets_csv_hashed.zip and _users_csv_hashed.zip contain a number of fields listed below

tweetid - tweet identification number
userid - user identification number (anonymized for users which had fewer than 5,000 followers at the time of suspension)
user_display_name - the name of the user (encoded as userid for anonymized users)
user_screen_name - the Twitter handle of the user (encoded as userid for anonymized users)
user_reported_location - the user's self-reported location (*)
user_profile_description - the user's profile description (*)
user_profile_url - the user's profile URL (*)
follower_count - the number of accounts following the user (*)
following_count - the number of accounts followed by the user (*)
account_creation_date - date of user account creation

Addeddate: 2018-10-21 11:45:51
Identifier: twitter-iran
Scanner: Internet Archive Python library 1.8.1
Year: 2018